A variety of different technologies rely upon accurate determination of position and orientation of a camera, referred to as the “pose” of the camera or “camera pose.” For example, robotics and computer vision systems, such as augmented reality and/or virtual reality systems, rely upon camera pose extensively. The ability to accurately determine and track camera pose allows a system to determine the position and orientation of 3-dimensional (3D) virtual objects with respect to the camera.
Conventional systems for estimating and tracking camera pose often rely upon photographic cameras. Photographic cameras are also referred to as “RGB” (Red, Green, Blue) cameras. Some systems utilize an RGB camera and one or more distinctive markers placed at, or affixed to, particular locations within the 3D environment to determine camera pose. Other systems such as Simultaneous Localization and Mapping (SLAM) based systems utilize RGB cameras and further create a map that is used for purposes of determining and/or tracking camera pose.
RGB cameras have a variety of characteristics that make them unsuitable for many applications that depend upon camera pose. For example, RGB cameras tend to be costly. RGB cameras also consume a significant amount of power. Many low power devices such as mobile phones, drones, and certain wearable devices like smart glasses operate primarily on battery power making power consumption an important consideration. RGB cameras are largely unsuitable for use in low power devices.
Other aspects of RGB cameras further limit the usefulness of the devices for camera pose estimation and tracking. For example, RGB cameras have relatively low frame rates that make the cameras sensitive to motion blur. Motion blur may lead to camera pose tracking failure. Frame-by-frame marker detection in RGB images is computationally expensive. The relatively low dynamic range of RGB cameras makes recognizing objects in very bright or in very dark 3D environments difficult.